System for capturing a document in an image signal

ABSTRACT

Devices, methods, and software are disclosed for capturing a frame of image data having a representation of a feature. In an illustrative embodiment, a device includes an imaging subsystem, one or more memory components, and a processor. The imaging subsystem is capable of providing image data representative of light incident on said imaging subsystem. The one or more memory components include at least a first memory component operatively capable of storing an input frame of the image data. The processor is in communicative connection with executable instructions for enabling the processor for various steps. One step includes receiving the input frame from the first memory component. Another step includes generating a reduced resolution frame based on the input frame, the reduced resolution frame comprising fewer pixels than the input frame, in which a pixel in the reduced resolution frame combines information from two or more pixels in the input frame. Another step includes attempting to identify transition pairs comprising pairs of adjacent pixels in the reduced resolution frame having differences between the pixels that exceed a pixel transition threshold. Another step includes attempting to identify one or more linear features between two or more identified transition pairs in the reduced resolution frame. Another step includes providing an indication of one or more identified linear features in the reduced resolution frame.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to the prior-filed provisional patentapplication with Ser. No. 61/347,292, entitled “System operative forcapture of a frame of image data having representation of a feature”,filed May 21, 2010, the entirety of which is incorporated by referenceherein.

FIELD OF THE INVENTION

The present disclosure relates to digital devices in general and inparticular to a digital device having an imaging subsystem.

BACKGROUND

Digital devices having imaging subsystems, such as smart phones, digitalcameras, and portable data scanning terminals, may be used for capturingimage frames having representations of one or more features.

The availability of higher density image sensor arrays having anincreased number of pixels, while providing certain advantages, can alsopresent challenges. With image sensor arrays having increasing numbersof pixels, frames of image data captured with use of such terminals haveincreasing numbers of pixel values. While a greater number of pixelvalues generally allows capture of a frame having a higher resolution,the higher resolution can result in increased processing delays. Imagesensor arrays are available in monochrome and color varieties; colorimage sensor arrays also provide increased data relative to monochrome.

The discussion above is merely provided for general backgroundinformation and is not intended to be used as an aid in determining thescope of the claimed subject matter.

SUMMARY

Systems are disclosed that in various embodiments include devices,methods, and/or software for capturing a document in an image signal. Inan illustrative embodiment, a device includes an imaging subsystem, oneor more memory components, and a processor. The imaging subsystem iscapable of providing image data representative of light incident on saidimaging subsystem. The one or more memory components include at least afirst memory component operatively capable of storing an input frame ofthe image data. The processor is in communicative connection withexecutable instructions for enabling the processor for various steps.One step includes receiving the input frame from the first memorycomponent. Another step includes generating a reduced resolution framebased on the input frame, the reduced resolution frame comprising fewerpixels than the input frame, in which a pixel in the reduced resolutionframe combines information from two or more pixels in the input frame.Another step includes attempting to identify transition pairs comprisingpairs of adjacent pixels in the reduced resolution frame havingdifferences between the pixels that exceed a pixel transition threshold.Another step includes attempting to identify one or more linear featuresbetween two or more identified transition pairs in the reducedresolution frame. Another step includes providing an indication of oneor more identified linear features in the reduced resolution frame.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter. The claimed subject matter is not limited to implementationsthat solve any or all disadvantages noted in the background.

BRIEF DESCRIPTION OF THE DRAWINGS

The features described herein can be better understood with reference tothe drawings described below. The drawings are not necessarily to scale,emphasis instead generally being placed upon illustrating the principlesof the invention. In the drawings, like numerals are used to indicatelike parts throughout the various views.

FIG. 1 depicts a simplified, mixed perspective and diagram view of asystem that includes a digital device with an imaging subsystem, inaccordance with an illustrative embodiment.

FIG. 2 depicts a schematic block diagram of a digital device with animaging subsystem, in accordance with an illustrative embodiment.

FIG. 3 depicts a flowchart for a method, in accordance with anillustrative embodiment.

FIGS. 4-8 depict simplified views of various aspects of processing imageframes for detecting and capturing a document, in accordance withvarious illustrative embodiments.

FIG. 9 depicts a graphical user interface application window with agraphical rendering of a document captured from an image signal, inaccordance with an illustrative embodiment.

The drawings are not necessarily to scale, emphasis instead generallybeing placed upon illustrating the principles of various embodiments. Inthe drawings, like numerals are used to indicate like parts throughoutthe various views.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

FIG. 1 depicts a system 5000 for capturing a document 110 in an imagesignal, in accordance with an illustrative embodiment that includes amobile device 1000, depicted here in perspective view. FIGS. 1 and 2provide illustrative devices and systems that may be used for performingdocument capture methods, while FIG. 3 depicts an illustrativeembodiment of a document capture method 200. As shown in FIGS. 1 and 2,mobile device 1000 may include an imaging subsystem 400 that has animaging field of view 1240, that may be surrounded by a projectedillumination field 1260, and which is directed at target document 110,in this illustrative embodiment. Mobile device 1000 and/or system 5000in various embodiments may be illustratively enabled to capture adocument 110 in an image signal, as is further outlined in thesubsequent figures and in the following description.

Mobile device 1000 is depicted as a hand held style mobile computer inthe illustrative embodiment of FIG. 1, and may also take the form of asmartphone, a mobile phone, a tablet or netbook computer, a laptopcomputer, an e-book reader, an indicia scanning terminal, or any of awide range of other types of digital devices with imaging subsystems, invarious embodiments. In the illustrative embodiment of FIG. 1, mobiledevice 1000 includes user interface elements including trigger 1220,display 1222, pointer mechanism 1224, and keyboard 1226 disposed on ahand held housing 1014. Two of the keys on keyboard 1225 are designatedas a scan key 1227 and an enter key 1228, though which keys are used forsuch functions and where they are disposed on mobile device 1000 may bearbitrarily selected and may differ from the illustrative depiction inFIG. 1.

In various illustrative embodiments, system 5000 may be operative sothat a preview window 1101 with a streaming preview image 1111 may beshown on the display 1222 of digital device 1000, as depicted in FIG. 1.The streaming preview image 1111 may show a reduced resolution real-timepreview of the image frames being imaged by the camera or imagingsubsystem 400. The preview image 1111 may serve a variety of usefulpurposes in promoting successful document capture. For example, anillustrative document capture processing implementation may allow a userto see a streaming document image 110B showing how document 110 is beingimaged by mobile device 1000, and showing when target document 110 isrelatively flat, well-lit, and encompassed within the field of view 1240of the imaging subsystem 400. An illustrative document captureprocessing implementation may pre-select a document profile, or inanother implementation, may provide a document profile type drop-downmenu 1141 allowing the user to select a document profile type having aspecified size and aspect ratio.

Display 1222 in various embodiments can incorporate a touch panel fornavigation and virtual actuator selection in which case a user interfaceof mobile device 1000 can be provided by display 1222. User interfaceelements of mobile device 1000 can also be provided by configuringmobile device 1000 to be operative to be reprogrammed by decoding ofprogramming bar code symbols. In another embodiment, a mobile device maybe devoid of a display and can be in a gun style form factor. In variousembodiments, mobile device 1000 may itself constitute a system forcapturing a document 110 in an image signal, and in various embodiments,mobile device 1000 in combination with one or more external servers2000, 3000 (depicted in block diagram), which may be connected over anetwork 2500, may together serve as a system for capturing a document110 in an image signal. In the description herein, system 5000 may bedescribed as being enabled or configured for various features,characteristics, or functions; and in various embodiments this may referto mobile device 1000 alone, or in communication or cooperation withother elements of system 5000, being enabled or configured for thosefeatures, characteristics, or functions. Various elements of FIG. 1 arefurther described below, after FIGS. 2-8 are introduced.

FIG. 2 depicts a schematic block diagram of mobile device 1000 withimaging subsystem 400, in accordance with an illustrative embodimentthat coincides with that of FIG. 1. Mobile device 1000 includes imagingsubsystem 400, one or more memory components 1085, and a processor.Imaging subsystem 400 is capable of providing image data representativeof light incident thereon. The one or more memory components 1085include at least a first memory component, illustratively such as RAM1080, that is operatively capable of at least temporarily or transientlystoring an input frame of the image data, while other memory componentsmay be used in various embodiments. The processor 1060 may be incommunicative connection with executable instructions for enabling theprocessor 1060 for various steps. Those steps are illustrativelydepicted in FIG. 3, in accordance with an illustrative method 200, andcertain aspects of image processing involved in those steps areillustratively depicted in FIGS. 4-6. Other referenced elements of FIG.2 are introduced and various elements of FIG. 2 are further describedbelow, after FIGS. 3-6 are further described.

In various illustrative methods, system 5000 may capture a document 110in an image signal using an illustrative document detection and locationprocess which may include several stages, which are briefly introducedhere and described in further detail below, in accordance with variousillustrative embodiments. Different document types may each have aprofile which holds parameters such as its edge lengths, target colorbalance, etc., and these parameters in conjunction with a set of fixedparameters may be used in controlling the process. Different steps arepresented as follows in an illustrative embodiment.

First, a frame of image data may be taken from a DirectShow previewstream via a frame grabber, or a null renderer used as a frame grabber,for example, and re-sampled to a smaller size, referred to as athumbnail. Then, a loop may be entered that uses differing criteria todetect edges, and chooses the one that gives the best results. For eachiteration of the loop, the thumbnail may be searched along both thevertical and horizontal axes for changes in brightness and/or color thatcould signify a document edge. Up to two changes may be registered foreach horizontal and vertical pass, in an illustrative embodiment. Edgesegments may be built out of consecutive changes, and then segments thatprobably belong to the same edge may be grouped together. There may be amaximum of four groups as only two changes were registered along eachaxis, which should correspond to the four edges of the document.Straight line equations may be fitted to the four edge groups, and thensolved as pairs of simultaneous equations to give four corner points.These points may then be transformed from 2D into 3D to give theoriginal document's corner points in 3-dimensional space. The documentdimensions may then be checked. After this, the document's position maybe smoothed over a given period of time to remove some instability inthe solution. The document may be checked for excessive shadows, and itsposition and orientation may be used to determine visual hints that canbe given to the operator to show how to move mobile device 1000 toimprove the image quality. Finally, once the operator may initiate afinal document capture the resulting image may be checked for sharpnessbefore being displayed. Once it is displayed, further processing may becarried out to transform it into a rectangular shape, reduce vignetting,correct any color imbalance, and sharpen it, in accordance with thisillustrative embodiment.

For example, FIG. 3 illustrates a method 200 of operating a mobiledevice 1000 for identifying a document in an image signal, in accordancewith an illustrative embodiment. For example, processor 1060 may be incommunicative connection with executable instructions that enableprocessor 1060 for performing the steps of method 200. As shown in theillustrative embodiment of FIG. 3, method 200 includes step 201, ofreceiving an input frame from the first memory component, illustrativelysuch as RAM 1080. This is followed by step 203 of generating a reducedresolution frame based on the input frame, the reduced resolution framecomprising fewer pixels than the input frame, in which a pixel in thereduced resolution frame combines information from two or more pixels inthe input frame. Step 205 includes attempting to identify transitionpairs comprising pairs of adjacent pixels in the reduced resolutionframe having differences between the pixels that exceed a pixeltransition threshold. If such transition pairs are identified, then step207 may be performed, of attempting to identify one or more linearfeatures between two or more identified transition pairs in the reducedresolution frame. If such linear features are identified, this can befollowed by step 209, of providing an indication of one or moreidentified linear features in the reduced resolution frame.

Such linear features may be indicative of the edges of an image of arectangular document 110 in an image signal read by digital device 1000,and may be used to locate or isolate the image of the rectangulardocument 110 in the image signal. The image signal may be generated bythe imaging subsystem 400 of mobile device 1000, and may include animage of all or part of a target document 110 within a field of view1240 of the imaging subsystem 400. The image of the document 110 may notbe rectangular even if the target document 110 is rectangular, due toeffects such as skew based on perspective angle between the mobiledevice 1000 and the document 110, and various steps disclosed herein maybe used to compensate for or correct such skew. In various illustrativeembodiments of mobile device 1000 and method 200, techniques fordetecting and capturing a document may be typically capable of locatinga document in a real-time 320×240 video stream image signal in less than40 ms per frame, as an illustrative example, though other periods oftime greater and less than this value are applicable in differentembodiments. A target document 110 may include a document, a package,another type of substrate, or any substantially rectangular form havingan interpretable feature thereon, such as writing or other form ofindicia that is susceptible of being decoded or interpreted, forexample.

FIGS. 4-8 graphically depict aspects of a process for detecting oridentifying features, such as pixel transition pairs and linear featuresthat may be indicative of document edges, leading to identifying andcapturing an image of document 110 in a frame of data 301 imaged bymobile device 1000, in accordance with illustrative embodiments whichmay coincide with the illustrative method 200 of FIG. 3. FIGS. 4-8 arefurther described below.

In various embodiments, an image signal processing driver or applicationmay be incorporated in mobile device 1000. The image signal processingdriver may direct the process of loading frames of image data from theimage sensor array 1033 to a buffer memory component such as RAM 1080 tobe available to processor 1060. This may be in preparation for step 201of receiving an input frame from the first memory component, such as RAM1080. In an illustrative embodiment, mobile device 1000 can incorporatea version of DirectShow Media Management software from MICROSOFTCorporation of Redmond, Wash. In various other embodiments, mobiledevice 1000 can incorporate another video driver or other image signaldriver.

In an illustrative embodiment, step 203 may involve a frame of imagedata being taken from the DirectShow preview stream via a frame grabberor null renderer, and re-sampled to a smaller size, referred to as athumbnail. In this illustrative embodiment, the thumbnail is a reducedresolution frame based on the input frame of image data from the previewstream, the thumbnail comprising fewer pixels than the input frame.Generating the reduced resolution frame further may include sectioningthe input frame into groups of pixels, and for each of the groups ofpixels, averaging one or more properties of the pixels in the group ofpixels and generating an averaged pixel having the averaged propertiesof the group of pixels, in various illustrative embodiments. Each of thepixels in the thumbnail may therefore combine information from two ormore pixels in the input frame. As an illustrative example, the initialinput frame may be divided among sets of four pixels in 2×2arrangements, and each of those 2×2 pixel sets may be averaged orotherwise combined, according to brightness, color, or both, into asingle representative pixel in place of the original pixel set. Othersizes of pixel sets or methods of reducing the pixel count or amount ofdata in the initial image frame, such as combining sets of nine pixelsin 3×3 arrangements, or other groupings, may also be used in differentembodiments.

Various embodiments of a document capture process may be performed usingsubstantially all the groups of pixels in the input frame and applyingthe methods for generating substantially all the pixels in thethumbnail, where the process may include minor errors and discrepanciesthat may affect one of more pixels that do not materially alter theprocess at large. For example, minor errors may be caused by defectiveimaging array pixels, radiation effects, minor random downloadingerrors, or other minor effects that may interfere with a relativelysmall number of pixels, without materially affecting the process of eachgroup of pixels in the input frame being processed to generate eachpixel in the reduced resolution frame, with the understanding that“each” may be taken to mean “substantially each” rather than “absolutelyeach one”, according to various illustrative embodiments.

During the search, mobile device 1000 may be displaying the previewstream frames on display 1222. System 5000 may be operative so thatwhile preview stream frames are being generated and displayed, system5000 may be buffering higher resolution frames (e.g., 1600×1200,1280×1024) from which the preview stream frames can be derived.Responsively to a command being activated to save a frame or image (e.g.representation of a document), e.g. by actuation of a scan key 1227 ofkeyboard 1225, a buffered higher resolution frame corresponding to alower resolution frame processed for quality detection can be processedfor saving. The higher resolution frames in one example can have a pixelcount equal to a pixel count of image sensor array 1033 (e.g.,1600×1200), or a similar higher resolution (e.g., 1280×1024). Any stepof buffering an image data frame transiently or otherwise temporarilymay be understood to include storing the image data frame.

Additionally, the preview stream may already be a reduced resolutionversion of the full resolution imaging data being imaged by mobiledevice 1000, so that in this illustrative embodiment, the imaging drivergenerates a first reduced resolution image frame or series of frames,while a document identification application may generate a second-levelframe that has its resolution reduced further from the first-roundreduced resolution image frame from the imaging driver. In various otherembodiments, various other processes may be involved in generating thereduced resolution frame or frames, which may involve only a single stepof resolution reduction from the full resolution input frames, or anynumber and manner of resolution reduction steps.

For example, in one illustrative embodiment, mobile device 1000 may havean imager that merges a high resolution monochrome imager with arelatively lower resolution color imager such as a 640×480 pixel arrayVideo Graphics Array (VGA) color imager on the same imaging chip, as anillustrative example. In this illustrative embodiment, the color imagermay be used for a streaming preview display, while the high resolutionimager may be used for final document capture, for example.

While the thumbnail has reduced resolution and a reduced amount of datacompared with a higher resolution frame on which it is based, thisenables a reduced processing burden and a reduced duration of time forthe processor to perform subsequent steps involved in identifying andcapturing an image of a document, such as steps of identifying pixeltransition pairs and linear features.

As an illustrative example, a preview stream image frame of reducedresolution may be further reduced prior to image processing for documentfeature detection. For example, the input to a 640×480 VGA color imagermay be reduced in resolution to one quarter to generate an originalpreview image, with each group of 2×2 pixels in the original imagersignal being averaged or otherwise combined to generate a single pixelin the original preview image, in an illustrative embodiment. Thisoriginal preview image can illustratively be 320×240 R5G6B5, but can bereduced in resolution a second time, illustratively being re-sampled andconverted to 160×120 R8G8B8 before the detection process starts. Suchprocessing can provide various advantages, for example it may be fasterto search this smaller-sized 3 bytes-per-pixel image for edges. Also,such processing may remove some fine detail from the document that couldbe mistaken for an edge. Various other illustrative embodiments mayinclude only a single step of resolution reduction, and/or may use otherapproaches in reducing the resolution to generate a rapid edge detectionframe.

For example, in one illustrative embodiment, a mobile device 1000 may beable to generate reduced resolution linear feature detection thumbnailframes, and perform a full complement of linear feature detection anddocument capture processes on these reduced resolution linear featuredetection thumbnail frames, at a frame rate of around seven to tenframes per second, for example. Other reduced resolution linear featuredetection frame rates both higher and lower than this may be used inother embodiments.

Linear feature identifying steps may be performed in a loop after thethumbnail is created based on the input frame of image data. This isillustrated in FIG. 3, in which both identifying steps 205 and 207 maycontinuing iteratively looping before one or more of the sought featuresare identified. The number of each feature sought and the number ofloops performed for seeking the features may be different in differentembodiments. Different criteria may be used to detect edges, and onethat gives the best results may be chosen, in various embodiments.

For example, a step of identifying pixel transitions may involve thethumbnail being searched, iteratively in a loop, along scan lines inboth horizontal and vertical directions for changes in brightness and/orcolor between adjacent pixels, that could signify a document edge. Inother words, each adjacent pair of pixels along each scan line may beevaluated to evaluate whether the two pixels in the pair are differentenough, either in absolute terms or relative to other adjacent pixelpairs in the thumbnail, that they have some likelihood of representingan edge of a document.

This is illustratively depicted in FIG. 4, which depicts an image frame301 corresponding to a frame of image data taken from part or all of aframe of pixels of image sensor array 1033. For clarity, only a fewillustrative horizontal scan lines 311 and vertical scan lines 313 areshown in FIG. 4. In an illustrative embodiment, no more than twocandidate pixel transition pairs may be registered for each horizontalscan line 311 and each vertical scan line 313. FIG. 4 also depicts whitecircles to indicate identified horizontal pixel transition pairs 321along horizontal scan lines 311, and dark circles to indicate identifiedvertical pixel transition pairs 323 along vertical scan lines 313.

While a reduced resolution frame may be used for rapidly detecting andidentifying features, such as pixel transition pairs and linearfeatures, indicative of an imaged document in various embodiments, asindicated above, a higher resolution, frame e.g. of resolution equal tothe resolution of image sensor array 1033 or of other higher resolutioncan be processed for detecting indicative features, in variousembodiments in which processing power is sufficient for rapid featuredetection at higher resolution.

FIGS. 5-8 depict additional aspects of building on identified pixeltransition pairs to identify linear features such as edge segments,aligned groups of edge segments, and intersections of edge segmentgroups likely to define corners of a document image, and thenidentifying and compensating for perspective skew, to map an image of a2D document in a 3D space into a normalized 2D image of the document. Abrief overview of these aspects may be provided as follows. Variousfeatures indicative of a document and that may be detected as part of aprocess of detecting and identifying an image of a document, such aspixel transition pairs, edge segments, aligned edge segment groups, andedge segment group corner intersections, may collectively be referred toas document indicative features or indicative features, for example, inthat they indicate potential features of a document such as edges of adocument. Any of these features indicative of a document, as well as anidentified document fragment, a partially identified document, or anidentified document, may collectively be considered to be identifiedlinear features in the reduced resolution frame, in various illustrativeembodiments.

As FIG. 5 depicts, candidate linear features likely to be edge segmentsmay be identified out of consecutive pixel transition pairs, and thenedge segments that are aligned and show a likelihood of belonging to thesame edge may be grouped together. There may be a maximum of fouraligned edge segment groups as only two pixel transition pairs areregistered along each scan line or axis in this illustrative embodiment,which should correspond to the four edges of the document. Straight lineequations can be fitted to the four edge groups, and then solved aspairs of simultaneous equations to give four corner points, as depictedin FIG. 6. These points can then be transformed from 2D into 3D to givethe original document's corner points in 3-dimensional space, which isfurther described below with reference to FIGS. 7 and 8.

The document dimensions can then be checked after calculating itsphysical size by relating its apparent size in pixels to the knownoptical characteristics of the imager lens assembly 250, for example.After this, the document's position may be smoothed over a given periodof time to remove some instability in the solution. The document can bechecked for excessive shadows, and its position and orientation can beused to determine prompts, illustratively in the form of graphical iconsthat offer hints on improved positioning, that can be given to theoperator to show how to move the mobile device 1000 to improve the imagequality. Finally, once the operator initiates the final documentcapture, the resulting image can be checked for sharpness before beingdisplayed. Once it is displayed, further processing may be carried outto transform the resulting image into a rectangular shape, reducevignetting, correct any color imbalance, and sharpen it. In oneembodiment, the operator can then be permitted to save the image (e.g. arepresentation of a document).

An indicative feature detection process, in various embodiments, may useone or more of a variety of algorithms for detecting document indicativefeatures. One illustrative algorithm embodiment may involve scanningalong horizontal lines 311 and vertical lines 313 through the thumbnailand find those that contain exactly two significant pixel transitions,as indicated above with reference to FIG. 4. Another illustrativealgorithm embodiment may involve finding the first significant pixeltransition in each direction, starting from the centers of each of thefour edges and working towards the center of the thumbnail. A pixeltransition pair, rising above a pixel transition threshold, can beregarded as significant in absolute terms in various illustrativeembodiments, such as if the sum of the absolute difference between thethree color values of a pixel and those of its neighbor is greater thana specified value.

For example, each pixel may encode 24 bits of color data in an R8G8B8format, such that each of the red, green, and blue intensity of eachpixel may range from 0 to 255, and a threshold may be set of adifference of at least 64 in each of the three color components, as anillustrative example, or of at least 128 in each of the three colorcomponents, as another illustrative example. Other intensity thresholdsfor each color component, either lower, higher, or within this range,may also be used for pixel transition thresholds in other illustrativeembodiments. In another illustrative embodiment, color componentintensities may be multiplied and differences between the products ofthe color components in adjacent pixels may be evaluated for significantpixel transitions. In other illustrative embodiments, a pixel transitionpair can be regarded as significant in relative terms, such as byselecting the two pixel transition pairs with the greatest differencesalong each axis or scan line. In various embodiments, an evaluation thatcombines absolute and relative criteria may be used. In variousembodiments, both of the two algorithms described here may be utilizedwith several different transition values to see which one gives the bestend result. After each attempt at finding indicative features, theindicative feature detection process can carry on to see how far itgets. In case of any check failing, the next transition value and/oredge detection algorithm will be tried. The process can continue untileither a fully satisfactory result has been obtained, or when bothalgorithms have been used with all applicable transition values.

In practice, there may be different numbers of scan lines that find noedge even though there is an edge, lines that find an “edge” even thoughthere is no actual edge, and also lines that find both a horizontal andvertical edge.

As light and other conditions vary along each edge of a document beingimaged, an indicative feature detection process may work better on someparts of a target document than others. It may therefore not be uncommonto find that an edge has been broken up into several segments, each oneof which contains consecutive or aligned pixel transition points. A listof edge segments may be built up by comparing the first and second pixeltransition pairs of each horizontal and vertical line that have themwith the ones from the previous scan line (if any) to see if they areroughly similar and are moving in the same direction. If they are, thenthe points can be added to the current edge segment, otherwise a newsegment is started. Given margins of error may be accounted for, such asallowing a single outlier in each segment, for example, so as not tobreak up an edge segment that contains a single misidentified value. Atthe end of this process, segments with less than a specified minimumnumber of points may be discarded, leaving four sets of segments thatshould include each of the four edges.

In an illustrative embodiment, a least squares method may be used to fita straight line equation to each segment in each of the four sets. Eachset may then be examined individually, and the segments may be tested inpairs to see if they might belong to the same edge. To do this, theirtwo equations can be used to obtain a measure of the difference betweenthe two lines by summing the squares of the distance between the linesat a number of places over a range corresponding to the width or heightof the thumbnail as appropriate. If such measure is less than aspecified value, the segments can be merged. At the end of this process,merged segments with less than a specified minimum number of points canbe discarded, and the largest remaining merged segment in each set (ifany) is selected—this will eliminate segments that, for example,corresponded to a vertical edge when a horizontal edge was being lookedfor. A least squares method may be used again to fit straight lines tothe four segments.

FIG. 5 shows a possible result from a single line of at leastapproximately aligned edge segments 431 (each one shown as a thick linesegment) produced from the scan lines depicted in FIG. 4, togethergrouped and extended as representing merged segment line 433. Each ofedge segments 431 may be assembled from adjacent or consecutive sets ofvertical pixel transition pairs 323 that may be generated from verticalscan lines 313 across thumbnail 301. Edge segment 435 may also have beenassembled from consecutive sets of vertical pixel transition pairs 323generated from vertical scan lines 313, but in this illustrativeembodiment, edge segments generated by one particular orientation ofscan lines may be used to populate one orientation of edge segmentsgroups, as an error constraining technique. The at least roughly alignededge segments 431 may be merged, while the unaligned edge segment 435may be discarded because it was not aligned with the othervertical-scan-generated segments, and it was data-poor in terms ofnumber of pixel transition pairs detected in the vertical scans than themerged segments 431 (since it is closer to vertical and has more of aparallel component to the vertical scan lines and a lower vertical scanresolution, while the merged segments 431 are closer to horizontal andeach have more of a perpendicular component to the vertical scan linesand a higher vertical scan resolution). Similarly, horizontal scans maybe taken to detect only the segment groups that are closer to verticaland are read with higher resolution in the horizontal scans than thevertical, and similarly would be detected to be strongly out ofalignment with the orthogonal segments.

Individual segments may contain inaccuracies, but merging the segmentstends to produce a more accurate combined result, as illustrativelyshown in merged segment line 433 in FIG. 5. In other illustrativeembodiments, information from vertical and horizontal scans may be usedto detect and characterize any edge segments forming parts of any of thefour edges of a target document.

Sufficient processing for detecting, characterizing, and merging pixeltransition pairs and/or edge segments may produce four merged segmentlines 433, 437, 443, 447, and edge equations may be determined tocharacterize each of the four merged segment lines. Having obtained thefour edge equations, the edge equations may then be examined in pairs tocheck that their relative angles are at least close, such as within adesired margin, of 90°, in various embodiments. The check may beapproximate, and the angles may be recalculated more accurately lateron, in various embodiments. The purpose of this is to ensure that theedge building process has resulted in an image portion that couldrepresent the edges of a 2D rectangular form, as imaged in a 3D space.If the check is successful, the equations may be solved in pairs assimultaneous equations to give the four corner points 451, 453, 455, 457defined by the four intersections of each of the four pairs ofintersecting merged segment lines 433, 437, 443, 447, as depicted inFIG. 6.

The validity of these corner solutions may then be tested to a highdegree of accuracy. The corners were derived from a 2-dimensional imageof what should be a 2-dimensional document, but the document exists in3-dimensional space (3-space for short) and could have been rotatedalong the x, y or z axes. A certain amount of rotation along the z axisis not a problem as it does not affect the overall geometry of theshape, it just reduces the maximum possible document image size.However, any x and/or y axis rotation changes the camera's line of sighttoward the target document to a skewed perspective view, and changes thecamera's view of the rectangle into a non-rectangular quadrilateral withpotentially four different side lengths and corner angles. The 2-spacecorner angle check could be made stricter, or the aspect ratio of thesides calculated at this point, but doing this limits the amount of skewthat can be tolerated, and could also let through invalid cornersolutions, in various illustrative embodiments.

FIGS. 7 and 8 depict aspects of imaging such a skewed perspectivequadrilateral and interpreting a representation of a rectangulardocument from it. FIG. 7 depicts an illustrative example of this skewedperspective view quadrilateral 611 of an imaged document in an imageframe 601, where quadrilateral 611 is defined by four edges 631, 633,635, 637 and four corner points 651, 653, 655, 657 identified throughfeature detection processes as described above. Quadrilateral 611 couldrepresent a rectangle that is being viewed from below and slightly tothe left of its center, but it is also possible that it could representan invalid corner solution for a different rectangle, as shown by theinvalid solution line border 711 in FIG. 8. If very tight validationwere to be applied to the 2-space angles and side lengths, then thescope of interpreting a skewed image would be too tightly constrained,and the quadrilateral 611 would be rejected as being too different frombeing rectangular.

A process for identifying a rectangular document therefore may allow fora significant deviation of the angle of each of the four corners of acandidate quadrilateral from 90 degrees, and/or a significant deviationof the length of each pair of opposing sides of the quadrilateral frombeing equal, though still disqualifying candidate quadrilaterals withoverly broad deviations in their angles from 90 degrees or in the lengthof their opposing pairs of sides. Exactly how significant thesedeviations are allowed without disqualification may be different indifferent embodiments, and may be tunable within a single embodiment,for example.

To determine whether or not the 2-space corners could represent a flatrectangle of the required size and shape in 3-space, the cornercoordinates can be converted from 2-space to 3-space before doing anyfurther validation, in an illustrative embodiment. In the general casethe problem is underconstrained and there can be an infinite number ofsolutions, which would comprise all of the possible intersectionsbetween any plane and the infinite 4-sided pyramid defined by an apex atthe camera and whose four edges each pass through the apex and each ofthe four identified corner points 651, 653, 655, 657. In oneillustrative embodiment, this conversion may be performed usingnon-linear programming techniques, using various assumptions. In thisexample the following three assumptions can be made:

-   1. The 3-space coordinates really do represent a rectangle, and    therefore each 3-space corner angle is 90°.-   2. The center of the rectangle is at a fixed position. This means    that instead of four free variables (the z coordinates of each    corner), a lower number may be need such as only three or only two ,    and the remaining one or two can be calculated, for example.    Together with the first assumption, this prevents there being    infinitely many solutions and also prevents the degenerate solution    of all z coordinates being zero.-   3. The z coordinates of the corners are similar—in other words, the    rectangle is not excessively skewed relative to the camera. This    assumption provides for a set of reasonable starting values for the    non-linear solver so that it finds the solution more quickly, in    this illustrative embodiment. The distance of the rectangle to the    camera may be estimated from its apparent size and used as the    initial z coordinate for each corner.

Using the first assumption, an equation may be constructed whosevariables are the 3-space z coordinates of the three free corners andwhose result represents how far the four corner angles are from being90°, and using the other two assumptions this equation can be solved (inthe sense of minimizing the angle errors) using non-linear programming,in this illustrative embodiment. An example of an equation for thisillustrative embodiment may be constructed as follows, usingconventional algebraic notation, and with points in 3-space denoted by asingle Roman letter, points in 2-space by a single bold italic letter,lines as a pair of letters showing the end points of the line, andangles as a single letter with a circumflex (̂).

First, in this illustrative embodiment, the 2-space x and y coordinatesof the corners are converted to 3-space using a simple perspectivetransformation based on the assumed z coordinate of each point:

$\begin{matrix}{A_{x} = {{sA}_{x}A_{z}}} & {A_{y} = {{sA}_{y}A_{z}}} \\{B_{x} = {{sB}_{x}B_{z}}} & {B_{y} = {{sB}_{y}B_{z}}} \\{C_{x} = {{sC}_{x}C_{z}}} & {C_{y} = {{sC}_{y}C_{z}}} \\{D_{x} = {{sD}_{x}D_{z}}} & {D_{y} = {{sD}_{y}D_{z}}}\end{matrix}$

where s is a scaling factor derived from comparing the size of the2-space quadrilateral to the angular field of view of the camera. In thecase of having only three free variables, the z coordinates of points B,C and D can vary freely during the solution process, but the zcoordinate of point A is constrained by the second assumption above, andwill always be equal to 4R_(z) −B_(z)−C_(z)−D_(z) where R_(z) is theestimated distance of the center of the rectangle from the camera. Thenthe squares of the lengths of each side and each diagonal may becalculated as follows:

$\begin{matrix}{S_{AB} = \left( {A_{x} - B_{x}} \right)^{2}} & {+ \left( {A_{y} - B_{y}} \right)^{2}} & {+ \left( {A_{z} - B_{z}} \right)^{2}} \\{S_{BC} = \left( {B_{x} - C_{x}} \right)^{2}} & {+ \left( {B_{y} - C_{y}} \right)^{2}} & {+ \left( {B_{z} - C_{z}} \right)^{2}} \\{S_{CD} = \left( {C_{x} - D_{x\;}} \right)^{2}} & {+ \left( {C_{y} - D_{y}} \right)^{2}} & {+ \left( {C_{z} - D_{z}} \right)^{2}} \\{S_{DA} = \left( {D_{x} - A_{x}} \right)^{2}} & {+ \left( {D_{y} - A_{y}} \right)^{2}} & {+ \left( {D_{z} - A_{z}} \right)^{2}} \\{S_{A\; C} = \left( {A_{x} - C_{x}} \right)^{2}} & {+ \left( {A_{y} - C_{y}} \right)^{2}} & {+ \left( {A_{z} - C_{z}} \right)^{2}} \\{S_{BD} = \left( {B_{x} - D_{x}} \right)^{2}} & {+ \left( {B_{y} - D_{y}} \right)^{2}} & {+ \left( {B_{z} - D_{z}} \right)^{2}}\end{matrix}$

Now, error values that will decrease as the 4 corner angles becomecloser to 90° may be derived:

$\begin{matrix}{A_{ɛ} = S_{DA}} & {+ S_{AB}} & {- S_{BD}} \\{B_{ɛ} = S_{AB}} & {+ S_{BC}} & {- S_{A\; C}} \\{C_{ɛ} = S_{BC}} & {+ S_{CD}} & {- S_{BD}} \\{D_{ɛ} = S_{CD}} & {+ S_{DA}} & {- S_{A\; C}}\end{matrix}$

If any angle is 90° then by Pythagoras' theorem, the corresponding errorvalue will be zero. However, if the angle is not 90° then the cosinerule states that the error value, for example, for corner A will beequal to:

A _(ε)2·DA·AB·cos {circumflex over (A)}

which will tend to zero as A tends to 90°. The error value will alsotend to zero as DA and AB tend to zero, which will in turn tend to zeroas their corresponding z coordinates tend to zero. However, because ofthe constraints applied to the corners, such as because of the way A_(z)is defined in the current example, it can never be zero at the same timeas the other z coordinates, in this example. This prevents thedegenerate solution A_(z)=B_(z)=C_(z)=D_(z)=0.

Finally, the four error values may be combined into a single value thatthe solver can seek to minimize A method that will prevent a positiveerror from canceling out a negative error is needed, so the squares ofthe individual errors are summed:

Total_(ε) =A _(ε) ² +B _(ε) ² +C _(ε) ² +D _(ε) ²

Although it would be slightly faster to sum the absolute values of theerrors rather than their squares, doing so would introducediscontinuities into the equation's surface that could reduce theaccuracy of the solution, in this illustrative embodiment.

The solution method used in the illustrative example above is aniterative one that seeks to minimize the result of the above equation byrepeatedly changing the three free variables by a small delta value andseeing whether these changes improve the result or not. For eachiteration, it can calculate the total error, in one embodiment, up to 27times—this is because it will use each of the free variables as theyare, with the delta added, and with the delta subtracted (it may makeless than 27 calculations because it will not repeat ones that hadalready been made in the previous iteration). If a given set of changesdoes not improve the result, system 5000 can determine that a currentresult must be close to a local minimum and can therefore reduce thedelta for the next iteration. When either the delta reaches a specifiedminimum value or the number of iterations has reached a specified limitbefore the minimum delta value was reached, the solution process canterminate. This typically happens quickly because the equation's surfaceis continuous and relatively smooth, in this illustrative example.

The validity of the original corner solution can then be determined bytwo simple checks on the 3-space corners, in this illustrativeembodiment. First, the 3-space angles may be checked to be sufficientlyclose to 90°, and then the aspect ratio of the rectangle's sides may bechecked to be sufficiently close to the expected value. These tests canbe much stricter than corresponding checks on the 2-space quadrilateral.For example, in an illustrative implementation, the 3-space angles maybe checked to be in the range 90°±0.25°, and the aspect ratio to bewithin 2% of the expected value. The tests could be made even stricter,but then a document that was not relatively close to perfectly flatcould be rejected, in this illustrative implementation. If both testsare satisfied then processing can continue, in this example.

Various embodiments of an edge detection process may therefore includemeasuring angles between identified linear features, evaluating whetherthe measured angles are within a constrained skew value around 90degrees, and if they are, then identifying intersections of the linearfeatures as identified corner points. A skew value may be considered tobe equivalent to an angle error or an error value in the angles,indicating skew effects in the 3D imaging of a 2D 90 degree angle, invarious illustrative embodiments.

Smoothing of Results: The result values returned by the documentdetection process in this illustrative embodiment are the 2-D cornerlocations within the original image, the 3-D document distance, offsetfrom the camera axis and angles of rotation, the difference in aspectratio from the expected value, and the shadow amount. Because a smallchange in a single edge solution could make a significant difference tothe final 3-space solution, these results may be smoothed over aspecified time period by calculating moving averages, in thisillustrative embodiment. If the values suddenly change by more than aspecified amount then the new results may be temporarily ignored and theprevious ones returned, unless all four corners have changed—in thiscase, the moving averages may be reset because it may be assumed, inthis illustrative embodiment, that the camera (or the document) wasmoved quickly to a new position.

Additional Checks: Additional checks of image quality may also beperformed, such as an image shadow check and a camera movement check. Ashadow detection may for example be carried out by summing thumbnailpixel color values in groups, sorting the resulting list of values inascending order from darkest group to lightest, forming a histogramapproximately representing the original image brightness values, andexamining percentile values in the histogram, which may support areliable analysis between shadows and contrasting features intrinsic toa document, in this illustrative embodiment. A camera motion check maybe performed to prevent the user from attempting to capture a stillimage when mobile device 1000 is known to be in motion, in anillustrative embodiment. The motion check may, for example, comparegroups of pixel blocks from a current preview image to the previous one,and taking a change in any of the total red, green or blue values of thepixel groups have changed by more than a specified amount as anindication of excessive camera motion, in an illustrative embodiment.Various steps may be taken to prevent factors such as shadow or cameramotion from interfering with document image capture, including automatedsteps and/or operator hints.

Operator Hints: With the correct profile selected, an operator can aimimage sensor array 1033 or camera of mobile device 1000 at the document110. At this point, if a candidate document solution has passed all ofthe tests then it may be indicated that it probably does represent avalid document, in this illustrative embodiment. However, in anillustrative embodiment, system 5000 can be operative so that a finalset of checks may be made before the still image capture and saving(e.g. capture and saving of a representation of a document) ispermitted. In this illustrative embodiment, these checks ensure that thedocument satisfies certain criteria for quality imaging, such as thatthe document does not extend beyond the terminal's or camera's field ofview as can be defined by lens assembly 250 in combination with imagesensory array 1033, that the document is taking up a certain minimumamount of the field of view, and that the document is not so skewed sothat there could be depth of field issues or image degradation duringthe image transformation process. System 5000 may be operative so thatthe preview window can provide operator hint icons and/or othergraphical indications to help guide the user in positioning mobiledevice 1000 and/or document 110 to improve the imaging quality forcapturing a high-resolution image of document 110. A variety of operatorhint icons may be used to guide the user to take many types of actionsto improve imaging quality for document capture.

System 5000 in this illustrative embodiment may be operative so thatonce all imaging quality criteria are satisfied, the operator can beprompted to capture and save a still image (e.g. a representation of adocument, which representation can be included in a frame of imagedata). In one embodiment, system 5000 can be operative so that when theimaging quality checks are satisfied, mobile device 1000 mayautomatically image and save a high resolution copy of document 110, orprompt a user to enter an input to image and save a high resolution copyof document 110, such as by activating scan button 1227 on mobile device1000.

System 5000 can be operative so that when scan button 1227 is activated,actuation of the scan button 1227 results in a frame or image capturedsubsequent to the button actuation being saved or processed for saving.In various illustrative embodiments, a frame of image data captured andsubject to processing for saving may have a higher resolution than aframe subject to processing for quality detection (e.g., can have aresolution of e.g., 1600×1200 or 1280×1024). System 5000 can also beoperative so that when scan button 1227 is actuated, a buffered higherresolution frame, as described herein, corresponding to the lowerresolution frame subject to processing for quality detection is saved oris processed for saving of an image of a feature, e.g. a document. Thebuffered higher resolution frame can have a resolution equal to theresolution of image sensor array or another relatively higherresolution, e.g. 1600×1200 or 1280×1024, in various illustrativeembodiments. The saving of a frame or of an image e.g. a documentrepresentation can comprise writing the frame into a long term storagememory, e.g. memory 1084, which may be e.g. a flash memory or a harddrive, in various illustrative embodiments.

System 5000 may also be operative so that saving of frame or image (e.g.a document representation) may occur without actuation of scan button1227 responsively to a described quality check being completed with adetermination of acceptable quality and without also being responsive toan actuation of scan button 1227. Image data processed for saving can beimage data of a frame captured subsequent to the quality check beingcomplete or prior to completion of a quality check (as in the case of abuffered frame corresponding to the frame processed for quality checkbeing processed for saving), in various illustrative embodiments.

When a still image has been selected for capture and before thescreen-sized version of it is displayed, it can be checked forsharpness, in an illustrative embodiment. It may be very difficult tosee how sharp the image is by just looking at the preview screen-sizedversion, and this image sharpness check may compensate for that.

In an illustrative embodiment, once a sufficiently sharp still picturehas been captured, a screen-sized image of the document may be extractedfrom it and transformed so that it is rectangular and perpendicular. Forexample, FIG. 1 shows preview window 1101 with preview image 1111showing candidate document 110 as it may be captured by a documentcapture process; and FIG. 9 shows candidate document 110 as it has beenextracted from the preview image 1111 in preview window 1101 andtransformed so that it is rectangular and perpendicular, andtransformed, rectangular and perpendicular candidate document 110Bdisplayed by itself. During this process, color information may begathered that will be used to correct the image during the pre-saveprocessing. The image can be displayed on display 1222, and may befollowed by final pre-save processing steps before the document imagecan be saved, in this illustrative embodiment. This final pre-saveprocessing may illustratively involve any or all of the steps ofextracting the document image from the captured picture, transforming itusing 2×2 bilinear interpolation so that it is rectangular andperpendicular, reducing vignetting caused by the camera lens and theillumination subsystem 800, correcting the image brightness and colorbalance using the information gathered in the previous step so that itconforms to a target value held in the image profile, converting theimage to grayscale, sharpening the image such as with a convolutionfilter, and/or correcting color balance, in various illustrativeembodiments.

Vignette compensation: The pre-save processing may include compensatingfor vignetting by both the imaging lens and by the offset of theillumination source, acting in effect as a double vignettingcompensation, in an illustrative embodiment. This vignette compensationmay be combined in a single process with interpolation and perspectivecorrection in extracting the document from the image. The vignettecompensation for the illumination may incorporate the known distance ofthe offset between the illumination subsystem 800 and the imaging lensassembly 250 as well as the distance to the document.

The vignette compensation may adjust the value per pixel using a formulathat incorporates separate vignette compensation factors for the imagingoptics and for the illumination:

p′=p(1+ox ² y ² +fX ² Y ² /d)

Here, p′ is the new, compensated pixel value; p is the original pixelvalue; o is the amount of compensation required for optical effects ofthe imaging optics; x and y are the horizontal and vertical distancesrespectively in pixels of a given pixel from the center of the image; fis the amount of compensation required for the illumination source, suchas an LED lamp, and which may be set to 0 if the illumination assemblyis not used for a given image; X and Y are the horizontal and verticaldistances respectively in pixels of a given pixel from the point atwhich the centerline of the illumination source would intersect theimage; and d is the distance between the imaging lens and the documentbeing imaged. The pre-save processing thereby applies a vignettingcompensation to pixels of an image portion representing the rectangularform, wherein the vignetting compensation compensates for vignetting byboth the imaging lens and by the offset of the illumination source.

Image File Save: Once the pre-save processing is complete, the operatormay in various illustrative embodiments be prompted or permitted to savethe document image, e.g. by pressing the enter key 1228. System 5000 mayalso be operative so that saving of a document image occurs responsivelyto the described pre-save processing being complete without entry ofenter key 1228, i.e. without the saving further being responsive toactuation or enter Key 1228, in other illustrative embodiments. Savingmay be performed using the ImagingFactory component from MicrosoftCorporation, as an illustrative example. Saving an image may comprisewriting the image to a storage memory 1084, which may be e.g. a flashmemory or a hard drive in a selected image file format, in variousillustrative embodiments. System 5000 can be operative so that once adocument image has been successfully captured, a preview of the captureddocument can be displayed. A user prompt may be provided with the optionto save the captured document to a long-term storage medium, in variousillustrative embodiments.

Imaging Device Hardware: Returning to FIG. 2, additional detail isprovided on an illustrative mobile device 1000 with an image sensor asan exemplary hardware platform for support of operations describedherein. Mobile device 1000 may include lens assembly 250 which may beadapted for focusing an image of a document 110 located within a fieldof view 1240 on a target substrate, onto image sensor array 1033. Thecalculated distance of the document may optionally be used to manuallyor automatically set the focus of the lens to the distance of thedocument, in an illustrative embodiment. Field of view 1240 of mobiledevice 1000 and image sensor array 1033 can be defined by lens assembly250 in combination with image sensor array 1033. Image sensor 1032 mayinclude multiple pixel image sensor array 1033 having pixels arranged inrows and columns of pixels, associated column circuitry 1034 and rowcircuitry 1035. Associated with the image sensor 1032 may be amplifiercircuitry 1036 (amplifier), and an analog to digital converter 1037which converts image information in the form of analog signals read outof image sensor array 1033 into image information in the form of digitalsignals. Image sensor 1032 can also have an associated timing andcontrol circuit 1038 for use in controlling e.g., the exposure period ofimage sensor 1032, gain applied to the amplifier 1036. The noted circuitcomponents 1032, 1036, 1037, and 1038 may be packaged into a commonimage sensor integrated circuit 1040, in this illustrative embodiment.Image sensor integrated circuit 1040 may incorporate fewer than thenoted number of components, in various embodiments.

In one illustrative example, image sensor integrated circuit 1040 can beprovided e.g., by an MT9V022 (752×480 pixel array) or an MT9V023(752×480 pixel array) image sensor integrated circuit available fromMICRON TECHNOLOGY, INC. In one illustrative example, image sensorintegrated circuit 1040 can be provided by an AV2105 2 Megapixel Color(1600×1200 pixel array) available from ARECONT VISION. In oneillustrative example, image sensor integrated circuit 1040 can beprovided by an MTD001C12STC 2 megapixel color (1600×1200 pixel array)available from MICRON TECHNOLOGY, INC.

In one illustrative example, image sensor integrated circuit 1040 canincorporate a Bayer pattern filter, so that defined at the image sensorarray are red pixels at red pixel positions, green pixels at green pixelpositions, and blue pixels at blue pixel positions. Frames that areprovided utilizing such an image sensor array incorporating a Bayerpattern can include red pixel values at red pixel positions, green pixelvalues at green pixel positions, and blue pixel values at blue pixelpositions. In an illustrative embodiment incorporating a Bayer patternimage sensor array, processor 1060 prior to subjecting a frame tofurther processing can interpolate pixel values at frame pixel positionsintermediate of green pixel positions utilizing green pixel values fordevelopment of a monochrome frame of image data. In another illustrativeembodiment, processor 1060 may, prior to subjecting a frame for furtherprocessing, interpolate pixel values intermediate of red pixel positionsutilizing red pixel values for development of a monochrome frame ofimage data. In another illustrative embodiment, processor 1060 may,prior to subjecting a frame for further processing, interpolate pixelvalues intermediate of blue pixel positions utilizing blue pixel values.

In the course of operation of mobile device 1000, image signals can beread out of image sensor 1032, converted, and stored into a systemmemory such as RAM 1080. Mobile device 1000 may include one or morememory components 1085, which may illustratively include RAM 1080, anonvolatile memory such as EPROM 1082, a memory storage device 1084, andany of a variety of other types of memory components, in variousembodiments. Memory storage device 1084 may illustratively be or includea flash memory, a hard disc drive, any type of RAM, EPROM, EEPROM,DVD-ROM, CD-ROM, or other type of ROM, optical disc, magnetic disc,magnetic cassette, magnetic tape, or any other type of volatile ornon-volatile or removable or non-removable memory or data storagecomponents, in illustrative embodiments.

In various illustrative embodiments, mobile device 1000 may includeprocessor 1060 which can be adapted to read out image data stored inmemory 1080 and subject such image data to various image processingalgorithms. For example, one or more processors 1060 may illustrativelybe or include a central processing unit (CPU), a complex programmablelogic device (CPLD), an application-specific integrated circuit (ASIC),a field programmable gate array (FPGA), or any type of circuit capableof processing logic operations, in accordance with various embodiments.

Mobile device 1000 may include a system bus 1500 providing for busarbitration, that may include any of a variety of bus structures such asa memory bus or memory controller, a peripheral bus, or a local bus,using any of a variety of architectures, in various embodiments. Forexample, this may include a Peripheral Component Interconnect (PCI) orMezzanine bus, an Industry Standard Architecture (ISA) bus, an EnhancedIndustry Standard Architecture (EISA) bus, a Micro Channel Architecture(MCA) bus, a Video Electronics Standards Association (VESA) bus, orother bus architectures, in various embodiments. Mobile device 1000 mayinclude a direct memory access unit (DMA) 1070 for routing imageinformation read out from image sensor 1032 that has been subject toconversion to RAM 1080, in various embodiments. Other embodiments of thesystem bus architecture and/or direct memory access components providingfor efficient data transfer between the image sensor 1032 and RAM 1080may be encompassed in various embodiments.

Server 2000, server 3000, or other computing elements in an illustrativecomputing system of the present disclosure may similarly include anyvariety of one or more processors, one or more memory components, one ormore system bus or other data communication components, and othercomponents. A memory of system 5000 in different embodiments may includethe memories of any of mobile device 1000, server 2000, server 3000, orother elements in a computing and/or network environment.

Mobile device 1000 may include an illumination subsystem 800 forillumination of target area and projection of an illumination pattern1260, in various embodiments. Illumination subsystem 800 mayillustratively include one or more LED flash lamps, one or morecontinuous LED lamps, one or more xenon flash tubes, or otherillumination elements, for example. An illustrative mobile device mayalso be devoid of illumination subsystem 800, in various embodiments.Illumination pattern 1260, in the embodiment shown in FIGS. 1 and 2, canbe projected to be proximate to but larger than an area defined by fieldof view 1240, but can also be projected in an area smaller than an areadefined by a field of view 1240, for example.

In various embodiments, illumination subsystem 800 may also include anillumination lens assembly 300, as is shown in the embodiment of FIG. 2.In addition to or in place of illumination lens assembly 300,illumination subsystem 800 may include alternative light shaping optics,e.g., one or more diffusers, mirrors, and prisms. In use, mobile device1000 can be oriented by an operator with respect to a target areacontaining a document 110 bearing decodable indicia 120 in such mannerthat illumination pattern 1260 is projected on a decodable indicia 120.Decodable indicia 120 may include any type of characters, symbols, orother visually detectable features that are susceptible of beingdecoded. This may include characters and/or numerals that may be decodedby any of various optical character recognition (OCR) techniques, or a1D or 2D barcode symbol, as illustrative examples.

Referring to further aspects of mobile device 1000, lens assembly 250may be controlled with use of electrical power input unit 1202. In oneembodiment, an electrical power input unit 1202 may operate as acontrolled voltage source, and in another embodiment, as a controlledcurrent source. Illumination pattern light source assembly 500 may becontrolled with use of light source control circuit 1206. Light sourcecontrol circuit 1206 may send signals to illumination pattern lightsource assembly 500, e.g., for changing a level of illumination outputby illumination pattern light source assembly 500. Certain elements ofmobile device 1000, e.g., image sensor integrated circuit 1040 (andimage sensor array 1033), imaging lens 240, and illumination subsystem800 may be packaged into an imaging module 400 which may be incorporatedinto hand held housing 1014. In another illustrative embodiment, amobile device may not have an illumination subsystem.

Mobile device 1000 may include a number of peripheral devices,illustratively including trigger 1220 which may be used to make active atrigger signal for activating frame readout and/or certain decodingprocesses, in this illustrative embodiment. Mobile device 1000 may beadapted so that activation of trigger 1220 activates a trigger signaland initiates a decode attempt. Specifically, mobile device 1000 may beoperative so that in response to activation of a trigger signal, asuccession of frames may be captured by way of read out of imageinformation from image sensor array 1033 (typically in the form ofanalog signals) and then storage of the image information afterconversion into memory 1080 (which may buffer one or more of thesuccession of frames at a given time). Processor 1060 may be operationalto subject one or more of the succession of frames to a decode attempt.Mobile device 1000 in another illustrative embodiment may be devoid ofdecoding functionality.

In one illustrative embodiment the components of imaging assembly 900comprising lens assembly 250 and image sensor integrated circuit 1040may be duplicated in mobile device 1000 and each of the duplicateimaging assemblies 900 may be incorporated in hand held housing 1014 andeach may be connected with system bus 1500 and processor 1060 in themanner of imaging assembly 900 as shown in FIG. 2. In this illustrativeembodiment, one of the imaging assemblies 900 may be optimized for usein decoding decodable indicia and the other of the imaging assemblies900 may be optimized for use in capturing and saving frames of imagedata and representations of features within frames of image data.

For attempting to decode a bar code symbol, e.g., a one dimensional barcode symbol, in an illustrative embodiment, a processor of system 5000(e.g., processor 1060 of mobile device 1000) can process image data of aframe corresponding to a line of pixel positions (e.g., a row, a column,or a diagonal set of pixel positions) to determine a spatial pattern ofdark and light cells and can convert each light and dark cell patterndetermined into a character or character string via table lookup. Wherea decodable indicia representation is a 2D bar code symbology, a decodeattempt can comprise the steps of locating a finder pattern using afeature detection algorithm, locating matrix lines intersecting thefinder pattern according to a predetermined relationship with the finderpattern, determining a pattern of dark and light cells along the matrixlines, and converting each light pattern into a character or characterstring via table lookup, in this illustrative embodiment.

Mobile device 1000 may include various interface circuits for couplingvarious of the peripheral devices to system address/data bus (systembus) 1500, for communication with processor 1060 also coupled to systembus 1500. Mobile device 1000 may include interface circuit 1028 forcoupling image sensor timing and control circuit 1038 to system bus1500, interface circuit 1102 for coupling electrical power input unit1202 to system bus 1500, interface circuit 1106 for couplingillumination light source bank control circuit 1206 to system bus 1500,and interface circuit 1120 for coupling trigger 1220 to system bus 1500.Mobile device 1000 may also include a display 1222 coupled to system bus1500 and in communication with processor 1060, via interface 1122, aswell as pointer mechanism 1224 in communication with processor 1060 viainterface 1124 connected to system bus 1500. Mobile device 1000 may alsoinclude keyboard 1226 coupled to system bus 1500. Keyboard 1226 may bein communication with processor 1060 via interface 1126 connected tosystem bus 1500. Mobile device 1000 may also include range detector unit1208 coupled to system bus 1500 via interface 1108.

Mobile device 1000 may capture frames of image data at a rate known as aframe rate. A typical frame rate is 60 frames per second (FPS) whichtranslates to a frame time (frame period) of 16.6 ms. Another typicalframe rate is 30 frames per second (FPS) which translates to a frametime (frame period) of 33.3 ms per frame. A frame rate of mobile device1000 may be increased (and frame time decreased) by decreasing of aframe picture size. An illustrative embodiment may use an AV2105 imagesensor integrated circuit, in which a maximum resolution picture size(1600×1200) may be selected which may yield a frame rate of 24 FPS.Selection of an HDTV windowed picture size (1280×1024) may yield a framerate of 32 FPS. Using an MT9D001C12 STC image sensor integrated circuit,a maximum resolution picture size (1600×1200) may be selected which canyield a frame rate of 20 FPS. Selection of an SXGA windowed picture sizemay yield a frame rate of 28 FPS.

An illustrative physical form view of mobile device 1000 in oneillustrative embodiment is shown in FIG. 1. Trigger 1220, display 1222,pointer mechanism 1224, and keyboard 1226 may be disposed on a commonside of a hand held housing 1014 as shown in FIG. 1. Display 1222,pointer mechanism 1224, and keyboard 1226 in one embodiment may beregarded as a user interface, or user input/output components, of mobiledevice 1000. Display 1222 in one embodiment may incorporate a touchpanel for navigation and virtual actuator selection in which case a userinterface of mobile device 1000 can be provided by display 1222. A userinterface of mobile device 1000 may also be provided by configuringmobile device 1000 to be operative to be reprogrammed by decoding ofprogramming bar code symbols. A hand held housing 1014 for mobile device1000 may in another embodiment be devoid of a display and may be in agun style form factor.

The image processing steps described herein can be distributed amongmobile device 1000, servers 2000 and/or 3000 and one embodiment may beexecuted entirely by mobile device 1000. In such an embodiment, system5000 may be regarded as being provided by mobile device 1000.

A small sample of illustrative devices, systems, apparatuses, or methodsthat are described herein is as follows:

A1. A device comprising:

-   -   an imaging subsystem capable of providing image data        representative of light incident on said imaging subsystem;    -   one or more memory components, comprising at least a first        memory component operatively capable of storing an input frame        of the image data; and    -   a processor, enabled for:        -   receiving the input frame from the first memory component;        -   generating a reduced resolution frame based on the input            frame, the reduced resolution frame comprising fewer pixels            than the input frame, in which a pixel in the reduced            resolution frame combines information from two or more            pixels in the input frame;        -   attempting to identify transition pairs comprising pairs of            adjacent pixels in the reduced resolution frame having            differences between the pixels that exceed a pixel            transition threshold;        -   attempting to identify one or more linear features between            two or more identified transition pairs in the reduced            resolution frame; and        -   providing an indication of one or more identified linear            features in the reduced resolution frame.            A2. The device of A1, in which the processor is further            enabled such that generating the reduced resolution frame            further comprises sectioning the input frame into groups of            pixels, and for each of the groups of pixels, averaging one            or more properties of the pixels in the group of pixels and            generating an averaged pixel having the averaged properties            of the group of pixels.            A3. The device of A2, in which the processor is further            enabled such that generating the reduced resolution frame            further comprises sectioning the input frame into            first-stage groups of four pixels per group in a 2×2            arrangement and generating a first-stage reduced pixel            having the averaged properties of the first-stage group of            pixels, thereby generating a first-stage reduced resolution            frame; then sectioning the first-stage reduced resolution            frame into second-stage groups of four pixels per group in a            2×2 arrangement and generating a second-stage reduced pixel            having the averaged properties of the second-stage group of            pixels, thereby generating a second-stage reduced resolution            frame.            A4. The device of A2, in which the processor is further            enabled such that generating the reduced resolution frame            further comprises providing a greater number of color data            bits in the averaged pixels than in the pixels from the            input frame, such that the reduced resolution frame has less            reduction in resolution in color data than in pixel density.            A5. The device of A4, in which the imaging subsystem is            enabled for providing the image data with 16 bit pixels            having color data per pixel, and the processor is further            enabled such that generating the reduced resolution frame            further comprises averaging four adjacent of the 16-bit            pixels into one 24-bit pixel having 24 bits of color data            per pixel.            A6. The device of A1, in which the processor is further            enabled such that attempting to identify transition pairs            comprises measuring differences in at least one of overall            brightness and brightness of individual colors between the            adjacent pixels in the pairs.            A7. The device of A1, in which the processor is further            enabled such that attempting to identify the transition            pairs comprises performing transition search steps from one            or more transition search algorithms, including a first            algorithm comprising scanning horizontal and/or vertical            lines in the reduced resolution frame for transition pairs,            and a second algorithm comprising scanning lines going from            the edges toward the center of the reduced resolution frame            for transition pairs.            A8. The device of A7, in which the processor is further            enabled such that attempting to identify the transition            pairs further comprises evaluating potential transition            pairs under two or more potential values for the pixel            transition threshold.            A9. The device of A7, in which the processor is further            enabled such that attempting to identify the transition            pairs further comprises scanning each of the scanned lines            for two transition pairs per line.            A10. The device of A9, in which the processor is further            enabled such that attempting to identify the transition            pairs further comprises responding to a lack of two            transition pairs in a line by re-scanning one or more of the            lines with one or more altered values for the pixel            transition threshold.            A11. The device of A1, in which the processor is further            enabled such that attempting to identify one or more linear            features comprises connecting consecutive identified pixel            transitions into identified line segments, identifying sets            of line segments that are approximately in-line with each            other, and merging identified sets of in-line line segments            into identified linear features.            A12. The device of A1, in which the processor is further            enabled for measuring angles between identified linear            features, evaluating whether the measured angles are within            a constrained skew value around 90 degrees, and if they are,            then identifying intersections of the linear features as            identified corner points.            A13. The device of A1, in which the processor is further            enabled for evaluating whether four linear features with            four corner points can be identified, and if they are not            identified, then providing a user aid prompt signal via an            output component.            A14. The device of A1, in which the processor is further            enabled for evaluating whether four linear features with            four corner points can be identified, and if they are            identified, then attempting to calculate a conversion of a            two-dimensional quadrilateral defined by the four corners            into a rectangular form in three dimensions.            A15. The device of A14, in which the processor is further            enabled for converting an image portion defined by the            rectangular form in three dimensions into a corresponding            two-dimensional rectangular image portion and saving the            two-dimensional rectangular image portion in at least one of            the memory components.            A16. The device of A14, in which the processor is further            enabled such that after the rectangular form in three            dimensions is calculated, a high-resolution quadrilateral            image portion in the input frame corresponding to the            quadrilateral defined by the four corners in the reduced            resolution frame is converted into a corresponding            high-resolution two-dimensional rectangular image portion            and saved in at least one of the memory components.            A17. The device of A14, further comprising an illumination            source, and in which the processor is further enabled            applying a vignetting compensation to pixels of an image            portion representing the rectangular form, wherein the            vignetting compensation compensates for vignetting by both            an imaging lens of the imaging subsystem and by an offset of            the illumination source from the imaging lens.            A18. The device of A1, in which a series of input frames are            continuously provided by the imaging subsystem and buffered            by the first memory component while the processor performs            processing steps until a rectangle in three dimensions is            identified in at least one reduced resolution frame.            A19. A method, performed using one or more processors,            comprising:            receiving an input frame from a first memory component;            generating, using at least one of the processors, a reduced            resolution frame based on the input frame, the reduced            resolution frame comprising fewer pixels than the input            frame, in which a pixel in the reduced resolution frame            combines information from two or more pixels in the input            frame;            attempting, using at least one of the processors, to            identify transition pairs comprising pairs of adjacent            pixels in the reduced resolution frame having differences            between the pixels that exceed a pixel transition threshold;            attempting, using at least one of the processors, to            identify one or more linear features between two or more            identified transition pairs in the reduced resolution frame;            and            providing an indication of one or more identified linear            features in the reduced resolution frame.            A20. A computer-readable storage medium comprising            executable instructions capable of            configuring one or more processors for:            generating, using at least one of the processors, a reduced            resolution frame based on the input frame, the reduced            resolution frame comprising fewer pixels than the input            frame, in which a pixel in the reduced resolution frame            combines information from two or more pixels in the input            frame;            attempting, using at least one of the processors, to            identify transition pairs comprising pairs of adjacent            pixels in the reduced resolution frame having differences            between the pixels that exceed a pixel transition threshold;            attempting, using at least one of the processors, to            identify one or more linear features between two or more            identified transition pairs in the reduced resolution frame;            and            providing an indication of one or more identified linear            features in the reduced resolution frame.            A21. The computer-readable storage medium of A20, in which            the executable instructions are further capable of            configuring one or more processors such that:            generating the reduced resolution frame based on the input            frame comprises: sectioning a 640×480 input frame into            first-stage groups of four pixels per group in a 2×2            arrangement and generating a first-stage reduced pixel            having the averaged properties of the first-stage group of            pixels, thereby generating a 320×240 R5G6B5 first-stage            reduced resolution frame; then sectioning the first-stage            reduced resolution frame into second-stage groups of four            pixels per group in a 2×2 arrangement and generating a            second-stage reduced pixel having the averaged properties of            the second-stage group of pixels, thereby generating a            160×120 R8B8G8 second-stage reduced resolution frame;            attempting to identify transition pairs comprises: scanning            lines in the 160×120 R8B8G8 second-stage reduced resolution            frame and, for pairs of adjacent pixels in the lines,            evaluating whether the pixels in the pair of adjacent pixels            exhibit changes in brightness and/or color above a pixel            transition threshold, thereby generating identified            transition pairs; and            attempting to identify linear features between identified            transition pairs comprises: connecting consecutive            identified pixel transitions in the 160×120 R8B8G8            second-stage reduced resolution frame into identified line            segments, attempting to identify sets of line segments that            are within constrained error values of being in-line with            each other, merging identified sets of in-line line segments            into identified linear features, attempting to identify            corner points among the identified linear features,            attempting to identify a two-dimensional quadrilateral            defined by the corner points, attempting to calculate a            conversion of a two-dimensional quadrilateral defined by the            four corners into a rectangular form in three dimensions,            converting an image portion defined by the rectangular form            in three dimensions into a corresponding two-dimensional            rectangular image portion, and saving the two-dimensional            rectangular image portion in at least one memory component.

While the present invention has been described with reference to anumber of specific embodiments, it will be understood that the truespirit and scope of the invention should be determined only with respectto claims that can be supported by the present specification. Further,while in numerous cases herein wherein systems and apparatuses andmethods are described as having a certain number of elements it will beunderstood that such systems, apparatuses and methods can be practicedwith fewer than or greater than the mentioned certain number ofelements. Also, while a number of particular embodiments have beendescribed, it will be understood that features and aspects that havebeen described with reference to each particular embodiment can be usedwith each remaining particularly described embodiment.

1. A device comprising: an imaging subsystem capable of providing image data representative of light incident on said imaging subsystem; one or more memory components, comprising at least a first memory component operatively capable of storing an input frame of the image data; and a processor, enabled for: receiving the input frame from the first memory component; generating a reduced resolution frame based on the input frame, the reduced resolution frame comprising fewer pixels than the input frame, in which a pixel in the reduced resolution frame combines information from two or more pixels in the input frame; attempting to identify transition pairs comprising pairs of adjacent pixels in the reduced resolution frame having differences between the pixels that exceed a pixel transition threshold; attempting to identify one or more linear features between two or more identified transition pairs in the reduced resolution frame; and providing an indication of one or more identified linear features in the reduced resolution frame.
 2. The device of claim 1, in which the processor is further enabled such that generating the reduced resolution frame further comprises sectioning the input frame into groups of pixels, and for each of the groups of pixels, averaging one or more properties of the pixels in the group of pixels and generating an averaged pixel having the averaged properties of the group of pixels.
 3. The device of claim 2, in which the processor is further enabled such that generating the reduced resolution frame further comprises sectioning the input frame into first-stage groups of four pixels per group in a 2×2 arrangement and generating a first-stage reduced pixel having the averaged properties of the first-stage group of pixels, thereby generating a first-stage reduced resolution frame; then sectioning the first-stage reduced resolution frame into second-stage groups of four pixels per group in a 2×2 arrangement and generating a second-stage reduced pixel having the averaged properties of the second-stage group of pixels, thereby generating a second-stage reduced resolution frame.
 4. The device of claim 2, in which the processor is further enabled such that generating the reduced resolution frame further comprises providing a greater number of color data bits in the averaged pixels than in the pixels from the input frame, such that the reduced resolution frame has less reduction in resolution in color data than in pixel density.
 5. The device of claim 4, in which the imaging subsystem is enabled for providing the image data with 16 bit pixels having color data per pixel, and the processor is further enabled such that generating the reduced resolution frame further comprises averaging four adjacent of the 16-bit pixels into one 24-bit pixel having 24 bits of color data per pixel.
 6. The device of claim 1, in which the processor is further enabled such that attempting to identify transition pairs comprises measuring differences in at least one of overall brightness and brightness of individual colors between the adjacent pixels in the pairs.
 7. The device of claim 1, in which the processor is further enabled such that attempting to identify the transition pairs comprises performing transition search steps from one or more transition search algorithms, including a first algorithm comprising scanning horizontal and/or vertical lines in the reduced resolution frame for transition pairs, and a second algorithm comprising scanning lines going from the edges toward the center of the reduced resolution frame for transition pairs.
 8. The device of claim 7, in which the processor is further enabled such that attempting to identify the transition pairs further comprises evaluating potential transition pairs under two or more potential values for the pixel transition threshold.
 9. The device of claim 7, in which the processor is further enabled such that attempting to identify the transition pairs further comprises scanning each of the scanned lines for two transition pairs per line.
 10. The device of claim 9, in which the processor is further enabled such that attempting to identify the transition pairs further comprises responding to a lack of two transition pairs in a line by re-scanning one or more of the lines with one or more altered values for the pixel transition threshold.
 11. The device of claim 1, in which the processor is further enabled such that attempting to identify one or more linear features comprises connecting consecutive identified pixel transitions into identified line segments, identifying sets of line segments that are approximately in-line with each other, and merging identified sets of in-line line segments into identified linear features.
 12. The device of claim 1, in which the processor is further enabled for measuring angles between identified linear features, evaluating whether the measured angles are within a constrained skew value around 90 degrees, and if they are, then identifying intersections of the linear features as identified corner points.
 13. The device of claim 1, in which the processor is further enabled for evaluating whether four linear features with four corner points can be identified, and if they are not identified, then providing a user aid prompt signal via an output component.
 14. The device of claim 1, in which the processor is further enabled for evaluating whether four linear features with four corner points can be identified, and if they are identified, then attempting to calculate a conversion of a two-dimensional quadrilateral defined by the four corners into a rectangular form in three dimensions.
 15. The device of claim 14, in which the processor is further enabled for converting an image portion defined by the rectangular form in three dimensions into a corresponding two-dimensional rectangular image portion and saving the two-dimensional rectangular image portion in at least one of the memory components.
 16. The device of claim 14, in which the processor is further enabled such that after the rectangular form in three dimensions is calculated, a high-resolution quadrilateral image portion in the input frame corresponding to the quadrilateral defined by the four corners in the reduced resolution frame is converted into a corresponding high-resolution two-dimensional rectangular image portion and saved in at least one of the memory components.
 17. The device of claim 14, further comprising an illumination source, and in which the processor is further enabled applying a vignetting compensation to pixels of an image portion representing the rectangular form, wherein the vignetting compensation compensates for vignetting by both an imaging lens of the imaging subsystem and by an offset of the illumination source from the imaging lens.
 18. The device of claim 1, in which a series of input frames are continuously provided by the imaging subsystem and buffered by the first memory component while the processor performs processing steps until a rectangle in three dimensions is identified in at least one reduced resolution frame.
 19. A method, performed using one or more processors, comprising: receiving an input frame from a first memory component; generating, using at least one of the processors, a reduced resolution frame based on the input frame, the reduced resolution frame comprising fewer pixels than the input frame, in which a pixel in the reduced resolution frame combines information from two or more pixels in the input frame; attempting, using at least one of the processors, to identify transition pairs comprising pairs of adjacent pixels in the reduced resolution frame having differences between the pixels that exceed a pixel transition threshold; attempting, using at least one of the processors, to identify one or more linear features between two or more identified transition pairs in the reduced resolution frame; and providing an indication of one or more identified linear features in the reduced resolution frame.
 20. A computer-readable storage medium comprising executable instructions capable of configuring one or more processors for: generating, using at least one of the processors, a reduced resolution frame based on an input frame of image data, the reduced resolution frame comprising fewer pixels than the input frame, in which a pixel in the reduced resolution frame combines information from two or more pixels in the input frame; attempting, using at least one of the processors, to identify transition pairs comprising pairs of adjacent pixels in the reduced resolution frame having differences between the pixels that exceed a pixel transition threshold; attempting, using at least one of the processors, to identify one or more linear features between two or more identified transition pairs in the reduced resolution frame; and providing an indication of one or more identified linear features in the reduced resolution frame.
 21. The computer-readable storage medium of claim 20, in which the executable instructions are further capable of configuring one or more processors such that: generating the reduced resolution frame based on the input frame comprises: sectioning a 640×480 input frame into first-stage groups of four pixels per group in a 2×2 arrangement and generating a first-stage reduced pixel having the averaged properties of the first-stage group of pixels, thereby generating a 320×240 R5G6B5 first-stage reduced resolution frame; then sectioning the first-stage reduced resolution frame into second-stage groups of four pixels per group in a 2×2 arrangement and generating a second-stage reduced pixel having the averaged properties of the second-stage group of pixels, thereby generating a 160×120 R8B8G8 second-stage reduced resolution frame; attempting to identify transition pairs comprises: scanning lines in the 160×120 R8B8G8 second-stage reduced resolution frame and, for pairs of adjacent pixels in the lines, evaluating whether the pixels in the pair of adjacent pixels exhibit changes in brightness and/or color above a pixel transition threshold, thereby generating identified transition pairs; and attempting to identify linear features between identified transition pairs comprises: connecting consecutive identified pixel transitions in the 160×120 R8B8G8 second-stage reduced resolution frame into identified line segments, attempting to identify sets of line segments that are within constrained error values of being in-line with each other, merging identified sets of in-line line segments into identified linear features, attempting to identify corner points among the identified linear features, attempting to identify a two-dimensional quadrilateral defined by the corner points, attempting to calculate a conversion of a two-dimensional quadrilateral defined by the four corners into a rectangular form in three dimensions, converting an image portion defined by the rectangular form in three dimensions into a corresponding two-dimensional rectangular image portion, and saving the two-dimensional rectangular image portion in at least one memory component. 